Task Rescheduling using Relational Reinforcement Learning

Generating and representing knowledge about heuristics for repair-based scheduling is a key issue in any rescheduling strategy to deal with unforeseen events and disturbances. Resorting to a feature-based propositional representation of schedule states is very inefficient and generalization to unseen states is highly unreliable whereas knowledge transfer to similar scheduling domains is difficult. In contrast, first-order relational representations enable the exploitation of the existence of domain objects and relations over these objects, and enable the use of quantification over objectives (goals), action effects and properties of states. In this work, a novel approach which formalizes the re-scheduling problem as a Relational Markov Decision Process integrating first-order (deictic) representations of (abstract) schedule states is presented. Task rescheduling is solved using a relational reinforcement learning algorithm implemented in a real-time prototype system which makes room for an interactive scheduling strategy that successfully handle different repair goals and disruption scenarios. An industrial case study vividly shows how relational abstractions provide compact repair policies with minor computational efforts.